Semi-Supervised Structure Learning

Discriminative learning framework is one of the very successful fields of machine learning. The methods of this paradigm, such as Boosting, and Support Vector Machines have significantly advanced the state-of-the-art for classification by improving the accuracy and by increasing the applicability of machine learning methods. Recently there has been growing interest to generalize discrimative learning methods to handle structured labels. For example labeling a word sequence with a part of speech sequence or labeling a word sequence with a parse tree. A variety of learning methods have been generalized to the structured case including logistic regression, perceptron (voted and dual), boosting, SVMs and kernel logistic regression (See [1] for a review on this line of research). These techniques combine the efficiency of dynamic programming methods with the advantages of the state-of-the-art learning methods. Here we are interested in semi-supervised learning of structured label classification. An initial investigation of semi-supervised learning in the structured case is given in [2]. In discriminitive learning, one is interested in learning a mapping from an input x ∈ X to an output or response y ∈ Y. In the multi-class case, this can be formulized by constructing a linear function of the form F (x,y;w) = 〈w,Ψ(x,y)〉 and then mapping a given input x to the label f(x) defined as follows.